Constant cost of the computation-unit in efficiency graphs

Here we show how to ensure a constant cost for the computation-unit in graphs depicting the number of (sequential) computation-units at different problem sizes. This is needed for a meaningful evaluation of scalability and efficiency, specially for distributed computations where it is an assumption of the measurement. We report empirical evaluation with ADOPT revealing that the computation cost associated with a constraint check (commonly used – and assumed constant – in ENCCCs evaluations) actually varies with the problem size, by orders of magnitude. This flaw makes it difficult to interpret such skewed graphs. We searched for methods to fix this problem and report a solution. We started from the hypothesis that the variation of the cost associated with a constraint-check is due to the fact that the most inner cycles of some common constraint solvers like ADOPT do not consist of constraint checks, but of processing search contexts (i.e., other data structures). We therefore propose computation-units based on a basket of weighted constraint-checks and context processing operations. Experimental evaluation shows that we obtain a constant cost of the computation-unit, proving the correctness of our hypothesis and offering a better methodology for efficiency and scalability evaluation.